MEMORANDUM 


To: Mary Scuriock, Oregon Stream Protection Coalition 

From: Chris Mendoza, Mendoza Environmental LLC, Olympia, WA. 

Subject: General comments on Oregon Compliance Monitoring Survey results 

Date: 20 December 2018 

My name is Christopher Mendoza and I have been a participant in the 
Washington Department of Natural Resources (WDNR) forest practices 
compliance monitoring program for the past eight years. 1 represent several NGOs 
in Washington's environmental community (Washington Environmental Council, 
Washington Forest Law Center, Conservation Northwest, Wild Fish Conservancy, 
Olympic Forest Conservation Organization). I also serve on WDNR Cooperative 
Monitoring Evaluation and Research (CMER) committee that oversees the 
development and implementation of studies designed to test the effects of forest 
practice rules on aquatic resource Including fish habitat (e.g. riparian and stream 
conditions, water quality, steep unstable slopes, roads, etc.). WDNR's compliance 
monitoring program also has an advisory committee comprised of stakeholders 
including WA state forest practices regulatory agencies (Washington Departments 
of Ecology, WA department of Fish and Wildlife), Local Tribes, Timber companies, 
"small" family forestland owners, and Federal regulatory agencies (US Fish and 
Wildlife Service). All meetings are public and follow open public meeting laws in 
accordance with Washington State. 

Scope of this review : You have asked me to comment on the concerns you 
raised with the Oregon Board of Forestry in September, 2018 when the most 
recent ODF Compliance Audit was presented to the Board. In my opinion, you 
have correctly identified the key issues, although there are additional problems 
revealed by the appendices on methods which are discussed below. 

V 

1. Forest Landowner Volunteer Compliance Monitoring is Bias by Defauit 

Any credible forest practices compliance monitoring program must be 
administered by an agency that has the authority to access ANY and ALL forest 
landowner's property with adequate notice (in WA that's one week's notice) to 
guard against biased results from a selective, nonrandom sample. As you correctly 
point out, randomly selecting harvest units / sites to sample for compliance is 
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that selec tion process bias tax audit results? Without State agency authority to 
"access an? au91t forest practices on unbiased, randomly selected harvest units at 
the agency's discretion, ODF's Compliance Audifjesults are biased in the direction 
of fulf compliance while grossly underrepresenting non-compliance. The 
Compliance Audit report admits this to some extent in stating thatr^ 
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Moreover, PNI (private non-industriaf) landowners may not know the difference 
between a 'full enforcement Investigation and legal decisions on compliance' and 
being audited when It comes to volunteering. Both non-nonresponses and those 
contacted but unwilling to volunteer for the audit would also contribute to biased 
results. Would you volunteer knowing the state carries legal enforcement 
authority for gross violations? 


In Washington state (WAC 222-08-160(4)), the Compliance Monitoring Program 
must answer whether forest practices are being conducted in compliance with 
the rules and to provide statistkally sound biennial audits and reports to the 
Forest Practices Board. Further, 'Sample sue estimation is based on attaining a 
margin of error of */- 5% for the statewide compliance proportion for riparian 
and road activities.' See 
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In order to guard against landowner biased results, the WA protocol for 
Compliance Monitoring directs: 'Give the landowner a notification call with the 
dates that you wiff be reviewing their application,' and 'The landowner n%ay 








attend Che assessment; and they can ckarlfy elements of the f PA. However, they 
cannot be part of the decision-making process for determining the compliance of 
thefr activities' 
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2. Statistical Reporting Methods for Aggregating Artlfidally Inflate Results 

Tables 3,4 and S depict a rtificlallv inflated compliance rates resulting from 
methods that are fatally flawed by dividing the total number of non-compliance 
validated 'on the ground” by the total number {thousands In many cases) of 
potential rule applications that were never visited^ then usln^ that result as % 
compliance (Table 3 'overall com liance' row one '62S / 25^00 ■ 98%). This Is 
^analogous to state law enforcement offers detecting 200 out of 1000 motorist 
speeding on a major interstate (e.g. over 60 mph) and then dividing the 200 that 
. were noncompliant by the 100,000 motorists that used the same interstate, but 
were never checked for speed { 200 / 100/)00 « 100 % compliance), instead of by 
the lOOO they did check for speed {200 /1000 ■ 80% compliance, 20% non- 
compliance). The Report goes on using the same flawed method when breaking 
down overall compliance rates by administrative area (Table 3), landowner class 
(Table 4), and Rule division (Table S) all of which are statistically Inaccurate and 
misleading. Only after a sample site has been determined adequate for each 
forest practices rule, (by a qualified statistician) can one infer the results to the 
general population being sampled, and never by dividing the number of sites 
actually checked for compliance on the ground by the total number of the 
potential applications for that rule (see above). 

If oof's goal is to report statistically robust result on compliance rates for forest 
practices rules, or other sub<ategories {e.g. harvest units), they must first and 
foremost determine adequate field sample sl:e(s) required to Infer to the general 
'bbpulatlon not field sampled, as opposed to ODF's flawed method of dividing the 
f I eld 'Sa mgled by the total nufrihfi f of £C^'s that werTnot ftel^r 

sampled (e.g. usin 10 00 • S 000 FPAs as numerator). The OOF report gives a 
couple of examples of how this approach should wor, ough the sample size Is 
too small to statistically infer to the larger population, by simply reporting the 
total number of sites that were actually field*checked for compliance monitoring. 
In reporting on culvert sizing to the 50-year flow. On page 15 OOF reports: 



A notable finding with Division 625 rules centers aix)und culvert sizing and 50-year 
peak flows. The official ODF guidance states that culvert sizing should be 
performed using the methods expi'essed in Foi'est Practices Technical Note Number 
6 (Tech Note 5, 2002). Size is based upon the contributing drainage area to the 
sti'eam ci'ossing mi^, and mapped 50-year peak flow at the location of the stream 
crossing (cfs/mi2). Sixtj'-two percent of culverts were adequately sized to pass the 
50 - 3 ^ear peak flow (OAR 629-625-0320(2(a))). 

And on page 17 ODF reports: 

Compliance with culvert sizing requii'ements (OAR 629-625-0320(2(a))) has varied 
over the four years of study. The calculated compliance i*ate in 2017 was 62%. For 
comparison, this rate was less than 90% in 2013 (85%) and 2016 (73%), but 
compliance was 96% in 2014. This variability may be affected by sample size. In 
2017, thirteen stream culverts wei'e assessed for the ability to pass the 60-year flow. 

The above examples report culvert sizing more accurately for compliance rates, 
for that specific rule requirement, by only using culverts that were actually field- 
checked for compliance. In this case only 62% of culverts that were actually 
checked "were adequately sized to pass the 50-yearNpeak flow." Note that ODF in 
this case -- and only in this case -- did not then divid^the total number of culverts 
field-checked by the total number of culverts on forestlands applied to all FPAs 
(e.g. using 1,000 culverts as numerator would yield a compliance rate of 99%). 
Doing so would have artificially inflated the culvert sizing compliance rate, much 
like ODF has done for reporting compliance rates for other rules and harvest unit 
level compliance (See ODF report, Tables 3, 4, and 5). 

Another example would be to convert ODFs until level compliance for sediment 
delivery to streams by only using the number of harvest units that were actually 
sampled. The ODF report states (page 17): 

Sediment delivery to Waters of the State was unevenly distributed between harvest 
operations. Thirty-six units delivered sediment. The remaining 35 units that 
contained Waters of the State did not deliver sediment to those waters, and 29 units 
had no Waters of the State. 

This equates to a compliance rate of 49% (36/71) when excluding the 29 units 
that should be screened out because they contain no Waters of the State. 
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However, OOF lists compliance for Harvest units at *'99%'' because once again the 
Peld.sampled units were Inappropnately divided by the total number of harvest 
units, induding those that were not field checked. Table 3 listing compliance by 
nrte dMshn exemplifies how compliance rates become artificiallv inflated by 
placing the number of rule oppikotlons that were never field checked for 
compliance in the denominator {thousands in some cases) with Noneomplhnce In 
the numerator {1*364): 
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If OOF Intended to choose a method that selectively depicts the highest possible 
^q(pplianfA Htes-£or forest practices rules, culvert condition, road condition, 
harvest unit, etc., then ODF's reporting ''metho<fs^hrt?9>at mark.*'>lowever, a 
more cr^ibleTore^tpractic^cTn^an^eTnonItori^pr^ram would c^nsuIt with j 
t^uaJifM statisticians to detefa]gie,adeguate.samplftiize(sL.^ confidence ~ 
intervals {e.g. 95%) for specific rules being tested, and other related categories 
(e.g. harvest unit, rule division), before attempting to Infer the results to the 
larger landscape. And under no conditions of which I am aware has a statistician 
ever recommended dividing the actual field*sampled population by the total 
number of rule applications that were never visited for reasons stated above. 
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3. Additional shortcomings of ODF Compliance Monitoring Report 


Sediment delivery cannot reliably be estimated from visual observations. ODF 
reports visual observation of estimates of sediment volumes without using 
measurable criteria. While visual observations may be adequate to detect 
whether direct delivery of sediment-laden, turbid water is directly delivering to 
streams (e.g. muddy road ditch line water being conveyed via cross drains / 
culverts Into streams). It Is not an accurate method for estimating sediment 
volume. Methods exist that actually capture and measure water and sediment 
coming out of culverts, but visual observation is not one of them. 

Attempting to estimate direct Impacts to aquatic resources in the absence of a 

robust research and monitoring program that tests the effects of forest practices 

on fish and fish habitat Is not reliable. While ODF does not claim to be conducting 

effectiveness monitoring when It makes judgment calls during compliance 

monitoring about resource Impacts, the line between these two types of 
monitoring seems extremely blurred. 


Cause and effect relationships between forest practices and streams and their 
biota are normally tested through experimental design and Implementation. 


room e 


e prior example of visually estimating sediment conveyed 
through culverts directly into streams may be a good example of non-compliance 

with a road related rule, but the amount of sediment and effect that sediment has 

on fish and their habitat cannot be accurately assessed without further study. 

Moreover, unless the visual estimator happens to be present during a long- 

duration storm event, combined with logging truck haul traffic, they will likely 

under-detect and underestimate the amount of sediment being conveyed to 
streams. 


Recommendations for Improvement to Compliance Monitoring Reporting 

Third Party Review . The program would greatly benefit from having 
an independent scientific peer review of the framework, approach, and 
sampling methods currently being used by ODF. This would include a 

qualified statistician that could make recommendations on sample size(s) 
for those rules being tested. Washington State Department of Natural 
Resources recently had their compliance monitoring program go through 


independent scientific peer review at the University of Washington and has 
since revised their program. 


increase o ngoing oversight . Form a compliance monitoring program 
advisory committee comprised of experts representing stakeholders 
(landowners, state and federal regulatory agencies, tribal organization, 
environmental organizations, etc.) would help ensure that ODF results are 
widely supported and accepted. This would include holding regular public 
meetings updating the public on the current status, priorities and direction 

the ODF compliance program is pursuing with an expected timeline for 
results. 

^Parate compliance fro m effectiveness monitoring . ODF should develop 

and implement a discrete research and monitoring program to test the 

effects of forest practices on aquatic resources including water quality. 

Smce ODF recently adopted new riparian buffer rules, and has road rules in 

place, testing the direct effects those rules have on fish habitat and stream 

conditions (e.g. shade, steam temperature, sediment, wood loading) would 
more accurately estimate resource impacts. 


References 


Groom J.D, L. Dent, LJ. Madsen and J. Fleuret 


/iicAV . ui western 

Oregon (USA) stream temperatures to contemporary forest management 
Forest Ecology and Management 262 (2011) 1618-1629. 

HawksworthJ P_ Clements and J. Laine. 2017. Forest practices compliance 

udit. 2017 annual report. Oregon Department of Forestry. 

Oregon Department of Forestry. 2016. Oregon Department of Forestry private 

coSorT"’' “"'Pliance audit field guide for 





